Some data is needed repeatedly or frequently in a data processing system. A read or write operation with the data is significantly faster in a volatile memory device such as DRAM as compared to in a permanent storage device such as a hard drive.
Caching is the process of holding such data in a region of a memory device in the data processing system, as opposed to serving the data from a permanent storage device. Such a region of the memory device is called a cache memory, or simply, a cache. In a non-distributed caching environment, each computing node configures its own local cache and access to a cache is restricted to the computing node in which the corresponding region of the memory device is configured.
For example, often, several servers collaborate to service client requests in a data processing environment. For example, many application servers collaboratively service requests to perform operations on a back-end database. In such a configuration, numerous clients make database requests, and one of the collaborating application servers receives a particular request from a client. That application server attempts to service the database request using data in that application server's cache, if the required data is available in the application server's cache. When the data needed to service the request is present in that application server's cache, the application server is able to service the request much faster than when the data needed to service the request is not in that application server's cache. When the data needed to service the request is not in that application server's cache, the application server accesses the database for the needed data and services the request at a significantly slower speed.
Memcache is a distributed cache management mechanism whereby the amount of cache memory available to a collaborating server is increased by pooling memory from other collaborating servers. For example, if two application servers are collaborating in a memcache configuration, each server sets aside some or all of its memory for sharing with the other server. The shared memory is accessible to the collaborating servers as a cache pool called memcache. Thus, each collaborating server has access to a combined cache size that is much larger than the cache available in that server alone.
Each data object in the memcache has a key associated therewith. The collection of the keys forms a key-space. Each key can be hashed according to a hashing algorithm and the collection of all possible hash values according to the algorithm forms a hash space, whether or not an actual key hashes to a possible hash value in the hash space.
Memcache uses a consistent hashing scheme to distribute key-value records across multiple memcache servers. The consistent hashing scheme treats the hash space as a fixed circular ring. Each server node is assigned one or more random values on the ring. A server node is responsible for all keys in the hash-space that correspond to its position in the ring and its predecessor's (or successor depending on the implementation) position in the ring. The memcache clients use the hash of the key to identify the target server and route a request to the appropriate server node directly. Each server is responsible for its own cache that is participating in the memcache and is completely independent of other servers with no shared state and synchronization between servers. In other words, each server has visibility into a portion, or segment, of the memcache that spans the server's own cache. When a request for an object comes to a server, if the requested object exists in the server's own cache segment, the server responds with the data of the object from the server's own cache. Otherwise, the server has a cache miss.
For this operation, a request has to be specifically directed to a specific target memcache server in the hash ring. A client-side library handles the sending of a client's request to a designated target server in a hash ring. Depending upon the hash value of the key of the requested object, the library selects one of the memcache servers in a given hash ring as a target server, and sends the request with the key to that target server. If the requested object exists in the segment of memcache associated with that target server, the target server responds with the data of the requested object from that segment of the cache. Otherwise, the target server has a cache miss. The client receives the cache miss information and accesses the data from another storage location outside the memcache.